Evaluating and Predicting Actual Test Coverage

This thesis proposes a new notion of semantic coverage in formal testing: actual coverage. It is defined for test case and test suite executions, as well as for sequences of their executions. A fault is considered to be completely covered if an execution showed its presence, and it is considered partly covered if an execution increased the confidence in its absence. Actual coverage can be used to evaluate a test process after it has taken place, but we also describe how to predict actual coverage in advance. To support these estimations, a probabilistic execution model is introduced. We derive efficient formulae for both the evaluation and the prediction of actual coverage, making tool support feasible. We show that for an infinite number of executions our measure coincides with an existing notion of semantic coverage, called potential coverage. This notion, however, does not deal with the fact that in practice only a finite number of executions will be performed. With actual coverage it is possible to predict the actual coverage of any given number of test case or test suite executions. An extensive detailed example is provided to demonstrate the applicability of our measure.